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ABSTRACT 

This report examines the effects of exposure to various elements of a civics curriculum on civic 
participation, two forms of political knowledge, internal political efficacy, political cynicism, 
news elaboration, discussion elaboration and various forms of interpersonal and mediated 
political communication behaviors. The data are based on a longitudinal study of high school 
students in a challenged large urban school district in Ohio. Two approaches to instruction are 
contrasted: stimulating political communication by discussing media sources and engaging in 
political debate; and rote learning of traditional civics content. Both approaches correlated 
negat/ve/y with civic outcomes, but there could be several interpretations of that correlation. 



BACKGROUND 

The decline of participation in politics among young people in America has been of 
concern to both scholars and professionals (Delli Carpini, 2004; Zukin, Keeter, Jenkins, Andolina, 
& Delli Carpini, 2006). Their concerns have renewed a focus on the impact of civics teaching as 
an important agent of political socialization. In recent years, exposure to a civics curriculum has 
been shown to produce positive effects on political socialization outcomes (McDevitt & Kousis, 
2006; Meirick & Wackman, 2004; Simon & Merrill, 1998), even though previous research had 
dismissed civics classes as largely ineffective (Langton & Jennings, 1968; Niemi & Junn, 1998). 
While advances have been made in scholars’ understanding of how civics teaching can 
positively impact today’s youth, questions still remain. 

'Various curricular programs can have effects not only on the children who participate, 
but also on the children’s parents through increased discussion and media use at home 
(McDevitt & Chaffee, 2000; McDevitt & Kiousis, 2006; McDevitt, Kiousis, Wu, Losch, & Ripley, 2003; 
Simon & Merrill, 1998). Other research has highlighted the importance of communication in the 
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classroom as an important consideration for the effectiveness of the curriculum (Campbell, 2005; 
Torney-Purta, Barber, & Wilkenfeld, 2007). The impact of these communication effects, both 
inside and outside of the classroom, is an important consideration that should be further 
examined when evaluating the long-term effects of civic education programs. 

Another way to improve our current understanding of civics courses would be to expand 
the scope of political socialization outcomes. Factual political knowledge is a commonly 
employed outcome, but political knowledge is a broader concept than simple recall of 
disconnected facts (e.g., Eveland, Marton, & Seo, 2004). Specifically, this study incorporates a 
measure of knowledge structure density (KSD) to assess the structural aspect of political 
knowledge. KSD examines the extent to which students perceive political concepts as related or 
connected, whereas factual knowledge examines what bits of political information students 
know. Beyond knowledge, several important political beliefs and cognitions are known to be 
empirically tied to political participation among adults, and thus might be important 
socialization outcomes among adolescents. These variables, which generally have not been 
examined in longitudinal political socialization research, include political efficacy, cynicism, 
news-elaboration and discussion-elaboration (Cappella & Jamieson, 1995; Delli Carpini, 2004; 
Eveland, 2004). 

This report examines the effects of exposure to various elements of civics teaching on 
civic participation, two forms of political knowledge, internal political efficacy, political cynicism, 
news elaboration, discussion elaboration and various forms of interpersonal and mediated 
political communication behaviors. The data are based on a longitudinal study of high school 
students in a single, urban school district in Ohio. Outcomes are measured one year after 
exposure to the curriculum to better determine its long-term impact. We utilize multilevel 
modeling to assess the effects of macro-level variables describing individual schools, courses in 
the curriculum, and specific teaching behaviors within classes taught by specific teachers, as 
well as individual-level variables which may serve as moderators and/or mediators of the effects 
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of the macro-level variables. With respect to the content of civics instruction, we contrast two 
strategies of instruction: 1) those emphasizing the process of communication through using and 
evaluating politically-relevant media sources and engaging in debate and discussion regarding 
politics; and 2) those emphasizing rote learning of traditional civics content such as lengths of 
terms of office, constitutional amendments, legislative process, and the constraints on and 
expansion of suffrage. Results suggest that within this school district, contrary to many findings, 
the impact of schools and classroom activities is weak and often negative. 

METHOD 

Research Setting 

The present study was conducted in high schools within the Columbus Public School 
(CPS) district, which serves the urban portion of the Columbus, Ohio metropolitan area. The 
Columbus Metropolitan Statistical Area includes over 1 .7 million people, and 718,477 individuals 
reside in the city of Columbus (US Census Bureau, 2006). The CPS district represents a unique 
population for study not only because of its urban location but also because of its highly diverse 
demographic makeup and the struggling nature of its schools at the time of our study. 

According to the Census Bureau, 73.9% of Columbus city residents identify themselves as white, 
while 12.4% of residents identify themselves as black. The 56,151 students that attended CPS in 
2006, however, vary quite dramatically from the city population estimates. Overall, 62.5 % of CPS 
district students are black and 29.2% are white (Ohio Department of Education, 2006). During the 
2005-2006 school year, the average graduation rate for CPS district high schools was 68.5%, and 
73.9% of students were classified as “economically disadvantaged." Academically, the CPS 
district was designated as being on “Academic Watch" due to failing to meet adequate yearly 
progress, in addition to meeting only 5 of 25 state indicators of success in the 2005-2006 school 
year. By comparison, the Upper Arlington school district, which serves a relatively wealthy 
suburban community contiguous to Columbus and part of the Columbus MSA, has been 
designated as “Excellent" and met all of the indicators of success in the 2005-2006 school year. 
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The demographic make-up of Upper Arlington school district is also quite different, with 90.8% of 
the students being white and none considered economically disadvantaged. 

Participants 

Data were collected from social studies teachers in all but one CPS high school in 2005. 
Surveys were distributed to teachers by the social studies department chairs at each school. Of 
the approximately 100 high school social studies teachers in CPS, 67 teachers completed the 
survey. The survey asked teachers in indicate which of 1 2 curriculum components they utilized in 
each of the social studies courses they taught at the time. A listing of these components can be 
found in the following measures section. 

For each of the teachers who completed our survey, the district provided us with their 
class rosters, including home mailing addresses of the students. Data were collected in 2005 from 
parents and students in these classes; however, given our interest in the long-term impact of the 
various classroom activities and the low response rate for students in 2005 (see below), only 
student data collected in 2006 will be utilized here. In autumn 2006, immediately following the 
Ohio midterm election (including a U.S. Senate race and a gubernatorial race), a local survey 
research firm was employed to: (a) re-contact the 202 students who completed our survey in 
2005 and for whom we also had parent data; (b) contact the 2005 non-respondent students for 
whom we had 2005 parent data only; and (c) contact a random sample of the 2005 students for 
whom we had neither parent nor student data but for whom we had 2005 teacher data. 
Students were offered $7 in return for completing the survey.' A total of 896 students were 
interviewed. For 90 of those respondents we had 2005 student, parent, and teacher data; for 1 20 
of the 2006 respondents we had 2005 parent and teacher data only; and for 686 of the 2006 
student respondents we had only 2005 teacher data. Respondent age ranged from 1 3 to 20 (M 



* When students were contacted, interviewers first asked to speak with the parent to assess the age of the student. If 
the student was 18 or older, the student was then approached for consent. For students younger than 18, parental 
consent was obtained before the student was approached for assent. 
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= 16.41, SD = 1 .20). Both genders were represented fairly evenly (45.6% male, 54.4% female). The 
most prevalent ethnic group was African American (54.9%), followed by Caucasian (36.4 %). 2 
Measures 

Independent variables at the classroom environment and course level. The primary 
independent variables under both course and classroom environment nesting were the 
classroom activities that teachers indicated were utilized in their courses prior to the 2005 
election campaign. In order to examine nesting at the course level, aggregates for each activity 
were created across all classrooms and teachers for a specific course. 

Six of the measured activities explicitly incorporated communication, while six were more 
focused on traditional, rote learning of civic information. The communication activities were 
summed to create a “communication score'' which reflects the number of communication 
activities utilized by a given teacher in a given course. The six communication activities were: 
analyzing news media content, analyzing political advertising messages, reading the newspaper 
or watching TV news for an assignment, debating political issues in the classroom, talking to 
family members about their political opinions, and debating a classroom issue and coming up 
with a binding resolution. 

The learning activities were summed to create a “learning score” which reflects the 
number of activities utilized by the teacher that focused on more traditional civics teaching. The 
six activities were: teaching the length of terms of public officials, teaching students how to 
register to vote, teaching how a bill becomes a law, teaching how groups have been excluded 
from political processes in the past, and teaching the amendments that make up the Bill of 
Rights. While the means for each individual teacher on these two variables were somewhat 



^ Caucasians were somewhat overrepresented in our sample, and African Americans were somewhat 
underrepresented in the sample relative to the demographic statistics released by CPS. However, our sample should 
not necessarily reflect all students in CPS but instead only those who were enrolled in a social studies course - for 
many students, an elective - in 2005. We examined the racial make-up of each school to see if our differences were 
based on over- or under-sampling in a given school. However, according to Table 1, it appears that out sample is 
fairly representative of the actual racial makeup in each of the schools. The ethnicity that was the majority in each 
school according to district reports was also the ethnicity of the majority of the respondents for that school in our 
sample. Thus, any bias comes from some schools being relatively underrepresented in our sample. 
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different, Figure 1 represents what the mean communication and learning scores were for each 
course taught by at least one of the 67 social studies teachers at CPS. As can be seen from the 
figure, the number and type of activities taught in each course varied greatly. 

Figure 1. Mean Communication and Learning Scores by Course 



Mean Number of Activies by Course Title 




□ Comm Scores 

□ Learn Score 



Independent variables at the school level. To examine effects of the school that students 
attended, data available from the district were utilized to assess differences among schools. The 
variables examined were the percentage of students who self-identity as being white, the 
percentage of students who are considered economically disadvantaged, and the graduation 
rate for the school. 
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Dependent variables. Dependent variables were internal efficacy, cynicism, news 
elaboration, discussion elaboration, civic participation, factual political knowledge and KSD. See 
Table 1 for complete item wordings and descriptive statistics. 

Table 1. Item Wording and Descriptive Statistics tor Dependent Variables 



Item wording 


Mean 


SD 


a/KR-20 


Internal Efficacy 


3.53 


.73 


.57 


1 consider myself to be well qualified to participate in politics when 1 turn 1 8. 

1 think 1 am better informed about politics than most people my age. 

1 feel 1 have a pretty good understanding of the important political issues facing our country. 


Cynicism 


3.61 


.75 


.57 


Elected officials almost never keep campaign promises. 
Politicians will say anything to get elected. 

The government wastes a lot of the taxpayer's money. 


News Elaboration 


6.24 


1.64 


.76 



I try to think about how the various stories I come across in the news connect with one another to form a big picture. 
Often when I come across something in the news I recall it later and think about it. 

I often mentally tie what 1 come across in the news to things I have seen or heard before. 

1 often think about how what I come across in the news relates to other things I know. 

I try to relate the things I come across in the news to my own personal experiences. 

Discussion Elaboration | 6.75 | 1 .89 | .66 

When I talk with others about something in the news, I usually think about the topic after the conversation is over. 

When I talk with others about something in the news, I often relate what they say to my own experiences. 

When I talk with others about something in the news, it often makes me think more about my own opinions and beliets. 
Participation (Yes or No, proportion of yes responses) | .28 | .25 | .62 

In the tall, did you do any volunteer work? 

In the tall, did you go to any sort of club meeting? 

In the fall, did you work on a community project? 

In the fall, did you go to a community or neighborhood meeting? 

In the fall, did you work for a social group or cause? 

Do you belong to any local associations such as a religious group, a political club or organization, a social club or 

organization, ora neighborhood group? 

Knowledge | 4.48 | 3.04 | .82 

Do you happen to remember which office Ted Strickland was running for. Governor or Senator? 

Do you happen to remember if Ted Strickland is a Republican or a Democrat? 

Do you happen to remember which office Mike DeWine was running for. Governor or Senator? 

Do you happen to remember if Mike DeWine is a Republican or a Democrat? 

Do you happen to remember which office Ken Blackwell was running for. Governor or Senator? 

Do you happen to remember if Ken Blackwell is a Republican or a Democrat? 

Do you happen to remember which office Sherrod Brown was running for. Governor or Senator? 

Do you happen to remember if Sherrod Brown is a Republican or a Democrat? 

Could you tell me which of the two candidates for Governor this year, Ken Blackwell or Ted Strickland, maintained a Pro- 
life stance on abortion? 

Could you tell me which of the two candidates for Senator this year, Mike DeWine or Sherrod Brown, supported a 

timeline for withdrawal from Iraq? 

Knowledge Structure Density | 3.41 | .65 | NA 

I'm interested in whether or not you think each of the following pairs of topics is related to one another. By related I mean 
whether or not the issues affect or are affected by one another. Please tell me how related you think two concepts are 
by telling me a number from 1 to 5, where "1" means "not at all related" and "5" means "very closely related." There are no 
right or wrong answers to these questions - simply give your first reaction and don't bother to think in too much detail 
about each pair of concepts. 

All possible combinations of the following six terms were presented: 

Terrorism 

Fuel prices 

Economy 

Taxes 

Education 

Environment 
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Mediating variables. Communication variables were conceptualized as mediators of 
class environment and course effects and political socialization outcomes. In particular, we 
examined the number of days in a given week respondents reported watching a local television 
news program, reading a print newspaper, and discussing politics with someone. 

RESULTS 

All models were estimated using HLM with Restricted Maximum Likelihood estimation. 
Robust standard error estimates were employed^. The individual level variables included in all 
models were the child’s age, gender, and whether or not the student identifies as white. We 
considered three different contexts, as portrayed in Figure 3 along with the typical default 
assumption of no clustering (Panel 1 of Figure 2). The first cluster was the specific school within 
the district, since significant variations in variables such as race and graduation rate existed 
between schools (Panel 2 of Figure 2). The second context was the specific class taught by a 
specific teacher (which will be referred to as the class environment from this point on), since 
there are both individual differences across teachers in the way they teach, but also a single 
teacher will likely include different content in the different courses s/he teaches (Panel 3 of 
Figure 2). The final context was the given course independent of the teacher, since some 
courses (e.g., specific civics courses) will have considerably different content from others (e.g., 
American history). In Figure 2, we represent this form of clustering by using different colors to 
reveal clusters of geographically dispersed students (Panel 4). 



^ Tables for all analyses are available in the appendix. 
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Figure 2 . Visual Representation ot Clustering Strategies tor Multi-level Models 
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School Effects 

Three outcome variables - factual knowledge, participation and cynicism - had 
significant variation across schools. The school-level predictors included in these models were 
the graduation rate of the school, the percentage of White students in the school, and the 
percentage of students in the school who were classified as economically disadvantaged. All 
three of these predictors were added to the equation for the intercept, which allows us to 
determine the effect that those variables have on the school mean for each outcome variable. 
In addition, the cross-level interaction - e.g., whether or not being white (individual-level 
variable) in a school with higher or lower percentages of white students (school-level variable) - 
was also examined. The cross-level interactions test whether context has an effect on the 
relationship between traditional individual-level predictors and the individual-level outcome. 

The results suggest that in schools with higher percentages of economically 
disadvantaged students, the mean score of factual knowledge was lower, mean participation 
score was lower, and the mean score for cynicism was higher. In schools with a higher 
percentage of white students enrolled, the mean knowledge score was lower, as was the 
participation score and the cynicism score. There was also a significant cross-level interaction 
between self-identifying as white and the percentage of white students enrolled at a school. As 
can be seen in Figure 3, the data indicate that white students report lower levels of cynicism 
when they are in schools with higher percentages of other white students; non-white students 
report higher cynicism when they are enrolled in schools with higher percentages of non-white 
students. 
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Figure 3. Cross-level Interaction tor Cynicism 




Nonwhite White 



White enrollment 25th percentile 
White enrollment 5oth percentile 
White enrollment 75th percentile 
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Environment Effects 

Results indicate that newspaper use, factual knowledge, KSD, participation and cynicism 
varied based on the environment (i.e., teacher-class intersection). The characteristics of the 
environment that were considered in these models were the number of communication 
activities and number of learning activities reported by each teacher for a given class. Results 
indicate that environments which tend to have higher amounts of learning activities are 
associated with lower levels of civic participation and cynicism. Exposure to communication 
activities positively predicts cynicism. For all other outcomes, only the individual level variables as 
opposed to the contextual level variables were predictive. The individual-level predictors 
effectively account for environmental differences, likely due to variations in these individual-level 
variables across environments. 

Course Effects 

Newspaper use, discussion frequency, factual knowledge, participation and discussion 
elaboration varied based on the course in which respondents were enrolled. Similar to the 
environmental effects, the characteristics of the course considered were the average number of 
communication activities and average number of learning activities reported by all teachers 
teaching a given course. Results indicate that the average number learning activities in a course 
is negatively related to factual knowledge, and all other effects can be attributed to individual 
differences. 

Direct Effects of Communication 

Neither the classroom activities nor the characteristics of the school were significantly 
related to the communication variables; however, because there was significant clustering by 
classroom and environment it would be inappropriate to examine the direct effects of the 
various forms of communication on the outcome variables using traditional OLS regression. The 
contextual level that had the strongest clustering effect for communication behaviors was the 
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course in which students were enrolled. Therefore, multi-level modeling was used with course as 
the contextual level, but only individual level predictors were entered into the model. Age, 
gender and self-identification as white were retained as controls in addition to the three 
communication behaviors: frequency of reading newspapers, watching local television news 
and political discussion. The results are summarized in Table 2. The results demonstrate that for 
every socialization outcome variable, with the exception of cynicism, at least one of the 
communication variables was a significant predictor. Frequency of discussion was positively 
related to every variable except for cynicism. How often individuals watched local television 
news was positively related to internal efficacy and participation. Newspaper use was positively 
related to internal efficacy and marginally related to news elaboration. 



Table 2. 

Direct Effect of Communication Behaviors on Sociaiizafion Outcomes Accounting for Course-Level Clustering 





KSD 


Facfual 

Knowledge 


Internal 

Efficacy 


Cynicism 


Civic 

Parficipafion 


News 

Elaboration 


Discussion 

Elaboration 


Intercept 


3.17 (.17)* 


-.20 (2.22) 


2.43 (.54)* 


2.93 (.62)* 


.05 (.13) 


6.00 (1.95)* 


8.17 (.95)* 


Age 


-.004 (.008) 


.28 (.15)# 


.03 (.03) 


.03 (.03) 


.01 (.01) 


-.05 (.11) 


-.12 (.05)* 


Gender 
(female = 1 ) 


.09 (.06) 


-.80 (.37)# 


.03 (.09) 


.12 (.04)* 


-.01 (.02) 


.16 (.07)* 


-.17 (.13) 


Whife 


-.06 (.04) 


.19 (.13) 


.08 (.04)# 


-.21 (.10)# 


-.05 (.05) 


-.25 (.13)# 


-.16 (.14) 


TV News 


.01 (.02) 


.06 (.04) 


.05 (.01)* 


.02 (.02) 


.01 (.003)* 


.03 (.04) 


.03 (.05) 


Newspaper 


-.01 (.01) 


.02 (.06) 


.07 (.02)* 


-.004 (.02) 


.01 (.01) 


.04 (.02)# 


-.001 (.02) 


Discussion 


.05 (.02)* 


.37 (.07)* 


.05 (.02)* 


.03 (.02) 


.02 (.01)* 


.23 (.02)* 


.23 (.02)* 



Note: Cell entries are HIM coefficients with robust standard errors in parentheses. 
* = p<.05, #=p<.10 
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DISCUSSION 

The purpose of this study was to examine the long-term influence of macro-level factors 
related to civics education that are associated with important socialization outcomes among 
high school students. In order to do so, we employed data linking schools, social studies courses, 
and social studies teachers with students one year after their exposure to these macro-level 
influences. This paper assessed the joint influence of the macro-level factors and select 
individual variables on traditional indicators of political socialization such as factual political 
knowledge and political participation. We also extended these outcomes to include various 
civic-relevant perceptions and attitudes such as internal efficacy, cynicism, and knowledge 
structure density; behaviors such as news use and political discussion; and cognitive activities 
such as news and discussion elaboration. 

Our first basic assumption was confirmed: macro-level contexts such as school, course, 
and classroom environment had important implications for socialization outcomes. Among the 
seven central socialization outcomes, three varied significantly by the specific school building in 
which respondents had been enrolled the prior year. Four of the seven outcomes varied 
significantly based on the combined class/teacher environment in which they were located the 
year before. And, three of the seven outcomes varied on the basis of the particular course in 
which they were enrolled the prior year. 

Broken down by socialization outcome rather than by contextual unit, we found that 
both factual political knowledge levels and participation varied significantly across school 
(within the same district), course, and classroom environment, with the greatest variation being 
for factual knowledge. Cynicism as an outcome was also affected by context, with significant 
variation across both schools and classroom environments. Knowledge structure density and 
discussion elaboration also each varied by one of the contexts - classroom environment for KSD 
and course for discussion elaboration. News elaboration and internal political efficacy did not 
vary by any of the contextual units. 
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Once it becomes apparent that these clusters - by school, by course, or by the teacher- 
class environment - continue to have implications for socialization outcomes a year later, it is 
incumbent upon us to identify the key variables within these contexts that may be exerting 
influence. With regard to schools, we identified three variables that we thought might be 
operating to affect the socialization outcomes - the ethnic composition of the school, the 
percentage of economically disadvantaged students in the school, and the school’s graduation 
rate. Our results clearly demonstrate that among these three school-level variables, the most 
important were the ethnic and economic makeup of the students. For the three dependent 
variables for which there was significant variation at the school level, these two school-level 
variables (but not graduation rate) were significant predictors. For factual political knowledge 
and participation, schools with higher proportions of white students scored lower. Districts with 
higher proportions of economically disadvantaged students scored lower on knowledge and 
participation. 

The results for cynicism were somewhat different, however. There was a cross-level 
interaction such that although white students were lower in cynicism, white students in schools 
with higher proportions of white students were more cynical than white students in schools with 
lower proportions of white students. By contrast, the most cynical non-white students were those 
in schools with low levels of white enrollment; the lowest levels of non-white student cynicism 
were in schools with high levels of white enrollment (Figure 3) . Schools with higher percentages 
of economically disadvantaged students produced students with higher levels of cynicism. 

Our results for the class environment and course were much more ambiguous. Despite 
finding significant variation across these contextual units comparable to the level in the school 
context, our two key variables for these contexts - communication-related curriculum 
components and rote learning-related curriculum components - seem to have little influence on 
our socialization outcome variables. In many cases, it appears that individual-level variables 
were able to account for differences in context. That is, our individual-level control variables of 
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age, gender, and ethnicity varied across class environment or course, and once these 
individual-level variations were controlled, class environment and course effects disappeared. 

Nonetheless, we did identify some instances of curricular influence, although often in a 
direction opposite our expectation. For instance, class environments with a greater number of 
learning components actually appear to have reduced levels of participation compared to 
those had fewer learning components. Courses with more learning components produced lower 
levels of factual political knowledge than those with fewer learning components. And, class 
environments with more communication components actually increased rather than decreased 
cynicism, although more learning components produced lower cynicism compared to class 
environments with fewer learning components. It is important to note that learning and 
communication components are not a zero-sum game; in fact, they are positively correlated at 
r = .65. The reasons for these unanticipated findings are unclear, and we do not have sufficient 
data to probe them further. We must leave it up to future research to ascertain the process 
through which these findings were produced, presuming they can be replicated in other 
samples. 

Our expectations regarding indirect effects of educational context variables on 
socialization outcomes through news media use and interpersonal political discussion went 
largely unsupported. There was no evidence for an influence of school context on any of the 
communication variables. There were significant variations in newspaper use by the class 
environment, but these differences were not affected by the two curriculum variables and the 
between-class environment variation was completely accounted for by the individual-level 
demographic controls. There were also significant variations across courses for newspaper use 
and political discussion, but these variations do not appear to be related to the curriculum 
variables and are largely accounted for by the individual-level demographic controls as well. 

Given these findings, it is highly unlikely that educational context effects are mediated 
through communication variables as we had expected. This does not mean, however, that 
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communication variables were unimportant. In fact, Table 3 reveals that communication 
variables likely play a central role in important socialization outcomes. Political discussion is a 
significant and positive predictor of six of the seven outcomes after controlling for 
demographics, other communication variables, and between-course variation. Newspaper use 
and television news use both account for significant increments in variance in two of the seven 
outcomes. Thus, communication outside the classroom should not be ignored as an important 
source of political socialization outcomes. 

Syvertsen, Flanagan, and Stout (2007) found primarily positive effects of classroom 
activities on democratic outcomes, although like us they also found fewer significant 
relationships than expected. One potential reason for the unexpected results is that the measure 
of civic content exposure was reported by the teacher rather that the student, which is the more 
common strategy (McDevitt & Kiousis, 2006; Meirick & Wackman, 2004). There are strengths and 
weaknesses of our approach. On the one hand, by having the teacher report the activities 
instead of the students, we avoid results being spurious due to the positive correlation between 
a student remembering a classroom activity and being affected by that activity. That is, with 
student self-reports it is possible (and likely) that two students in the same class at the same time 
will report experiencing somewhat different environments. This may be due to variations in 
absences, variations in attention to the content, or ability to recall. Each of these factors is a 
confound in the measure if the measure is taken from students, and thus a true “context” effect 
is not being observed, but instead an individual effect. 

On the other hand, neither are the teacher reports infallible. It is possible that teacher 
reports are confounded by a social desirability response bias or recall as well. That is, some 
teachers may have overreported the number of the activities they actually employ because 
they wanted to appear to be better teachers than they were; or, they merely may have 
misremembered. 
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However, Syvertsen et al. (2007) also utilized teacher reported activities and did not find 
the same negative effects of civic information on socialization outcomes as we did. The use of 
teacher reports instead of student reports could be accounting for the smaller and non- 
significant findings that were found in both studies, but it cannot account for our negative 
findings. 

Another potential explanation, and also where we differed from much prior research, is 
our sample. As we have already noted, the students in the CPS district represent only a subset of 
the population that is typical for socialization studies. Consequently, the schools were also quite 
different. Although the graduation rate of the schools was never a significant predictor of the 
dependent variables, we examined the relationship between graduation rate and the number 
of classroom activities that were reported. The analysis revealed that schools with lower 
graduation rates (which we would expect to produce weaker socialization outcomes) 
employed more classroom activities (which we would expect to produce stronger socialization 
outcomes). Together, each variable (unfortunately at different levels of analysis) could be 
suppressing the other. If such were the case, this could account for some of our findings 
because students who were in the weakest schools, for which we would expect the weakest 
effects of teaching, were also experiencing the most activities in their classroom. It is possible 
that teachers in poorly performing schools were in a sense trying (unsuccessfully) to compensate 
for the school environment. If so, then to conclude that exposure to more communication- 
related curriculum components was harming political socialization would be the equivalent of 
concluding that aspirin causes headaches because most instances of someone taking aspirin 
involves someone experiencing a headache. Ideally we would be able to tease out the 
interactive relationships between classroom environment and school factors, but our sample 
sizes at this level of analysis are inadequate to do so. 

In conclusion, this project reinforces the importance of examining the context in which 
information is presented, whether it is communication-based or knowledge-based. Although 
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questions still remain regarding the nature of the relationship between socialization outcomes 
and civics curricula programs, this project provides evidence that some samples may respond 
differently than what has previously been observed. Hopefully our findings can encourage 
researchers to continue to probe this issue and to implement multi-level modeling when asking 
questions about school effects. 
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Appendix A 

Correlation Table for Dependent, Mediating and Control Variables 





1 


2 


3 


4 


5 


6 


7 


8 


9 


10 


1 1 


12 


(1) Factual Knowledge 


























(2) KSD 


. 062 # 
























(3) Internal Efficacy 


259 *=^ 


. 152 ** 






















(4) Participation 


21 8 *=^ 


. 100 ** 


. 195 ** 




















(5) Cynicism 


-.020 


. 096 ** 


-.059 


-.026 


















(6) News Elaboration 


, 28 ** 


. 262 ** 


279 ** 


. 184 ** 


-.041 
















(7) Discussion Elaboration 


. 071 * 


. 227 ** 


. 234 ** 


. 162 ** 


.018 


. 699 ** 














(8) Local TV viewing 


. 142 ** 


. 140 ** 


245 ** 


. 122 ** 


.019 


. 155 ** 


. 124 ** 












(9) Newspaper reading 


. 161 ** 


.043 


. 262 ** 


. 162 ** 


-. 063 # 


. 202 ** 


. 142 ** 


. 183 ** 










(10) Discussion frequency 


. 232 ** 


212 ** 


. 269 *’^ 


231 ** 


.032 


. 287 ** 


. 288 ** 


. 275 ** 


. 300 ** 








(11) Age 


. 071 * 


.035 


-.045 


.003 


. 119 ** 


-.008 


.023 


.021 


. 058 * 


. 100 ** 






(12) Gender 
Female = high 


-. 120 ** 


. 083 * 


-.017 


.037 


.013 


.054 


.025 


-.046 


-. 079 * 


.036 


-.042 




(13) White 


.017 


-. 088 * 


-.005 


.007 


-. 144 ** 


-.054 


-.001 


.015 


-.056 


.048 


-.056 


-.034 



= p<.01, * = p<.05, # = p <.10 
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Appendix B 

Summary of Intraclass Correlations (ICC) and Significance of Level of Variance Accounted for in Random Effects (Empty) 
HLM Models 



Dependent Variable 


School 


Classroom 

Environment 


Course 




ICC 


p-value 


ICC 


p-value 


ICC 


p-value 


Local TV News Use 


.09% 


>.500 


.16% 


>.500 


.01% 


>.500 


Newspaper Use 


.64% 


.294 


4.53% 


.016 


1 0.57% 


.000 


Discussion Frequency 


.08% 


>.500 


.72% 


.323 


7.69% 


.000 


Factual Knowledge 


6.25% 


.000 


15.47% 


.000 


18.09% 


.000 


Knowledge Structure Density 


.01% 


>.500 


3.63% 


.037 


.01% 


>.500 


Participation 


1.59% 


.095 


1 .52% 


.085 


3.13% 


.015 


Internal Efficacy 


.59% 


.228 


.18% 


>.500 


.91% 


.234 


Cynicism 


3.99% 


.000 


5.73% 


.001 


.74% 


.178 


News Elaboration 


.01% 


>.500 


.11% 


>.500 


1 .77% 


.298 


Discussion Elaboration 


.02% 


>.500 


.02% 


>.500 


3.00% 


.021 



Note: p-value refers to whether or not there is significant variance in the dependent variable based on the Level 2 
group. 
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Appendix C 

School Contextual Model tor Factual Political Knowledge 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for ((3o) 








Intercept (yoo) 


8.76 (3.27) 


2.68 (9) 


.026 


Percent white (yoi) 


-.02(.007) 


-3.33 (9) 


.010 


Percent econ. Disadvantaged (yo2) 


-.06(.02) 


-3.62 (9) 


.006 


Graduation rate (yos) 


-.01 (.01) 


-.58 (9) 


.574 


Model for ((3i) 








Age (yio) 


.07 (.17) 


.39 (12) 


.700 


Model for (P2) 








Gender (y2o) 


-.67 (.21) 


-3.20 (12) 


.008 


Model for (Ps) 








White/non-white (yao) 


.68 (.39) 


1.73 (11) 


.111 


Percent white (yai) 


-.01 (.01) 


-1.43 (11) 


.180 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. between schools (Too) 


59.22 


7 


28.20 (p = .000) 


Var. in age (Tu) 


.22 


10 


27.363 (p = .003) 


Var. in gender (T22) 


.17 


10 


6.578 (p > .500) 


Var. in white/non-white (T33) 


.1 1 


9 


10.024 (p = .335) 


Var. within schools (G ) 


7.88 
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Appendix D 

School Contextual Model tor Civic Participation 



Fixed Effects 


Coefficient (SE) 


f(df) 


P 


Model for ((5o) 








Intercept (yoo) 


JAl (.22) 


3.45 (9) 


.008 


Percent white (yoi) 


-. 002 (. 001 ) 


-2.97 (9) 


.016 


Percent econ. Disadvantaged ( 702 ) 


-.005(.001) 


-3.28(9) 


.010 


Graduation rate (yos) 


-.001 (. 001 ) 


-1.02 (9) 


.335 


Model for (Pi) 








Age (y,o) 


-.004 (.012) 


-.35 (12) 


.750 


Model for (P 2 ) 








Gender ( 720 ) 


.024 (.023) 


1.05 (12) 


.375 


Model for (P 3 ) 








White/non-white ( 730 ) 


-.019 (.055) 


-.34(11) 


.769 


Percent white ( 731 ) 


.0007 (.001) 


.697 (11) 


.568 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. between schools (too) 


.194 


7 


14.964 (p = .036) 


Var. in age (in) 


.001 


10 


16.858 (p = .077) 


Var. in gender (T22) 


.003 


10 


12.321 (p = .263) 


Var. in white/non-white (133) 


.001 


9 


5.962 (p > .500) 


Var. within schools (o') 


.061 
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Appendix E 

School Contextual Model tor Cynicism 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Modei for ((?„) 








Intercept (yoo) 


2.080 (.54) 


3.83 (9) 


.005 


Percent white (yoi) 


-.005(.002) 


-2.32 (9) 


.045 


Percent econ. Disadvantaged ( 702 ) 


.008 (.004) 


2.1 1 (9) 


.064 


Graduation rate ( 703 ) 


.004 (.004) 


1.40 (9) 


.197 


Model for ((?i) 








Age ( 7 , 0 ) 


.045 (.02) 


2.48 (12) 


.029 


Model for (P 2 ) 








Gender ( 720 ) 


.069 (.07) 


.93 (12) 


.372 


Model for (P 3 ) 








White/non-white ( 730 ) 


-.430 (.13) 


-3.41 (11) 


.006 


Percent white ( 731 ) 


.008 (.003) 


2.56 (11) 


.027 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. between schools (t„„) 


.040 


7 


7.484 (p = .380) 


Var. in age (in) 


.0002 


10 


5.648 (p >.500) 


Var. in gender (T22) 


.026 


10 


16.870 (p = .077) 


Var. in white/non-white (133) 


.004 


9 


6.024 (p > .500) 


Var. within schools (o') 


.539 
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Appendix F 

Classroom Environment Contextual Model for Newspaper Use 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for ((?„) 








Intercept (yoo) 


.368 (1.15) 


.32 (91) 


.750 


Communication Score (yoi) 


-.085 (.06) 


-1.34 (91) 


.183 


Learning Score (joi) 


-.003 (.06) 


-.05 (91) 


.965 


Model for (Pi) 








Age (yio) 


.166 (.07) 


2.34 (93) 


.022 


Model for (P 2 ) 








Gender (y^o) 


-.361 (.17) 


-2.19 (93) 


.031 


Model for (P 3 ) 








White/non-white (yso) 


-.144 (.18) 


-.80 (93) 


.427 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. between environments (Xoo) 


3.321 


37 


V.777 (p = .434) 


Var. in age (xn) 


.015 


39 


39.114 (p = . 465) 


Var. in gender (X22) 


.047 


39 


37.239 (p > .500) 


Var. in white/non-white (X33) 


.086 


39 


30.372 (p > .500) 


Var. within environments ( 0 ^) 


4.199 
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Appendix G 

Classroom Environment Contextual Model for Factual Political Knowledge 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for ((?„) 








Intercept (yoo) 


.978 (1.84) 


.53 (91) 


.596 


Communication Score (yoi) 


-.110 (. 12 ) 


-.90 (91) 


.368 


Learning Score (joi) 


-.083 (.11) 


-.75 (91) 


.457 


Model for (Pi) 








Age (yio) 


.295 (.11) 


2.64 (93) 


.010 


Model for (P 2 ) 








Gender (y^o) 


-.518 (.22) 


-2.32 (93) 


.022 


Model for (P 3 ) 








White/non-white (yso) 


-.029 (.28) 


-.105 (93) 


.917 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. between environments (Xoo) 


1 1 .957 


37 


42.115 (p = . 259) 


Var. in age (xn) 


.044 


39 


42.731 (p = .314) 


Var. in gender (X22) 


.328 


39 


35.974 (p > .500) 


Var. in white/non-white (X33) 


.840 


39 


48.122 (p = .150) 


Var. within environments (cP) 


7.350 
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Appendix H 

Classroom Environment Contextual Model for Knowledge Structure Density 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for ((?„) 








Intercept (yoo) 


3.165 (.38) 


8.304 (91) 


.000 


Communication Score (yoi) 


-.011 (. 02 ) 


-.52 (91) 


.606 


Learning Score (joi) 


.007 (.02) 


.43 (91) 


.668 


Model for (Pi) 








Age (yio) 


.009 (.02) 


.39 (93) 


.699 


Model for (P 2 ) 








Gender (y^o) 


.110 (.06) 


1.99 (93) 


.049 


Model for (P 3 ) 








White/non-white (yso) 


-.080 (.05) 


-1.49 (93) 


.140 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. between environments (Xoo) 


.371 


37 


32.573 (p > .500) 


Var. in age (xn) 


.001 


39 


32.539 (p > .500) 


Var. in gender (X22) 


.027 


39 


45.402 (p = .222) 


Var. in white/non-white (X33) 


.011 


39 


31.595 (p > .500) 


Var. within environments (cP) 


.413 







Hutchens & Eveland 



29 I P a g 



CIRCLE Working Paper 65 



ww w. civicyouth. org 



Appendix I 

Classroom Environment Contextual Model for Civic Participation 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for ((?„) 








Intercept (yoo) 


.084 (.13) 


.64 (91) 


.521 


Communication Score (yoi) 


.002 (. 01 ) 


.46 (91) 


.646 


Learning Score (joi) 


-.018 (. 01 ) 


-3.20 (91) 


.002 


Model for (Pi) 








Age (yio) 


.014 (.01) 


1.67 (93) 


.099 


Model for (P 2 ) 








Gender (y^o) 


.015 (. 02 ) 


.74 (93) 


.461 


Model for (P 3 ) 








White/non-white (yso) 


-.016 (. 02 ) 


-.75 (93) 


.456 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. between environments (Xoo) 


.09 


37 


34.688 (p > .500) 


Var. in age (xn) 


.001 


39 


33.629 (p > .500) 


Var. in gender (X22) 


.004 


39 


44. 180 (p = .262) 


Var. in white/non-white (X33) 


.0001 


39 


37.342 (p > .500) 


Var. within environments (cP) 


.062 
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Appendix J 

Classroom Environment Contextual Model for Cynicism 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for ((?„) 








Intercept (yoo) 


2.806 (.40) 


7.02 (91) 


.000 


Communication Score (yoi) 


.067 (.02) 


2.91 (91) 


.005 


Learning Score (joi) 


-.054 (.02) 


-2.48 (91) 


.015 


Model for (Pi) 








Age (yio) 


.048 (.02) 


1.96 (93) 


.053 


Model for (P 2 ) 








Gender (y^o) 


.055 (.06) 


.862 (93) 


.391 


Model for (P 3 ) 








White/non-white (yso) 


-.200 (.07) 


-3.01 (93) 


.004 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. between environments (Xoo) 


.404 


37 


35.007 (p > .500) 


Var. in age (xn) 


.001 


39 


36.434 (p > .500) 


Var. in gender (X22) 


.054 


39 


40.167 (p = .418) 


Var. in white/non-white (X33) 


.022 


39 


42.601 (p = .318) 


Var. within environments (cP) 


.502 
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Appendix K 

Course Contextual Model for Newspaper Use 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for ((?„) 








Intercept (yoo) 


-1.070 (1.67) 


-.64 (11) 


.534 


Communication score (yoi) 


-.175 (.21) 


-.82 ( 11 ) 


.430 


Learning score ( 702 ) 


.052 (.12) 


.43(11) 


.678 


Model for (Pi) 








Age (yio) 


.294 (.11) 


2.60 (13) 


.022 


Model for (P 2 ) 








Gender ( 720 ) 


-.502 (.19) 


- 2.66 (13) 


.020 


Model for (Ps) 








White/non-white ( 730 ) 


-.097 (.09) 


-1.04 (13) 


.318 



Random Effects (Van Components) 


Variance 


df 


Chi-square 


Var. befween courses (x„o) 


.771 


5 


9.427 (p = .092) 


Var. in age (xn) 


.012 


7 


9.889 (p = .194) 


Var. in gender (X22) 


.107 


7 


9.465 (p = .220) 


Var. in whife/non-white (X33) 


.014 


7 


2.677 (p > .500) 


Var. within courses (a^) 


4.336 
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Appendix L 

Course Contextual Model for Discussion Frequency 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for ((?„) 








Intercept (yoo) 


.421 (1.53) 


.28 ( 11 ) 


.789 


Communication Score (yoi) 


-.151 (.21) 


-.71 (11) 


.491 


Learning Score (joi) 


.059 (.15) 


.40 (11) 


.695 


Model for (Pi) 








Age (yio) 


.235 (.10) 


2.432 (13) 


.030 


Model for (P 2 ) 








Gender ( 720 ) 


-.135 (.20) 


-.673 (13) 


.513 


Model for (Ps) 








White/non-white ( 730 ) 


.325 (.05) 


7.1 16 (13) 


.000 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. befween courses (x„o) 


2.068 


5 


1 1 .665 (p = .039) 


Var. in age (xn) 


.003 


7 


11.831 (p = .106) 


Var. in gender (X22) 


.191 


7 


9.795 (p = .200) 


Var. in whife/non-white (X33) 


.002 


7 


3.876 (p > .500) 


Var. within courses (a^) 


4.037 
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Appendix M 

Course Contextual Model for Factual Political Knowledge 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for (p„) 








Intercept (yoo) 


-8.839 (6.10) 


-1.45 (11) 


.175 


Communication Score (yoi) 


.140 (.25) 


.55 (11) 


.593 


Learning Score ( 702 ) 


-.395 (.14) 


-2.73 (11) 


.020 


Model for (pi) 








Age (yio) 


.960 (.37) 


2.60 (13) 


.041 


Model for (P 2 ) 








Gender ( 720 ) 


-.852 (.29) 


-2.99 (13) 


.040 


Model for (ps) 








White/non-whIte ( 730 ) 


.788 (.26) 


3.09 (13) 


.061 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. befween courses (Xoo) 


269.025 


5 


21.206 (p = . 001 ) 


Var. in age (xn) 


1.020 


7 


25.197 (p = .001) 


Var. in gender (X 22 ) 


.408 


7 


13.499 (p = .060) 


Var. in whlfe/non-whlte (X33) 


.434 


7 


10.921 (p = .141) 


Var. within courses (a^) 


8.634 
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Appendix N 

Course Contextual Model for Civic Participation 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for (p„) 








Intercept (yoo) 


.060 (. 12 ) 


.49 (11) 


.637 


Communication Score (yoi) 


-.027 (.04) 


-.70(11) 


.497 


Learning Score ( 702 ) 


-.014 (.03) 


-.49 (11) 


.633 


Model for (pi) 








Age (yio) 


.025 (.01) 


2.56 (13) 


.024 


Model for (P 2 ) 








Gender ( 720 ) 


-.020 (. 02 ) 


-.927 (13) 


.491 


Model for (ps) 








White/non-white ( 730 ) 


-.047 (.04) 


-1.19 (13) 


.339 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. befween courses (x„o) 


.016 


5 


7.225 (p = .203) 


Var. in age (in) 


.0001 


7 


7.317 (p = .397) 


Var. in gender (X22) 


.001 


7 


5.294 (p > .500) 


Var. in whife/non-white (X33) 


.013 


7 


10.379 (p = .167) 


Var. within courses (a") 


.062 
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Appendix O 

Course Contextual Model for Discussion Elaboration 



Fixed Effects 


Coefficient (SE) 


tldfl 


P 


Model for (p„) 








Intercept (yoo) 


7.138 (1.33) 


5.35 (11) 


.000 


Communication Score (yoi) 


-.027 (.24) 


-. 12 ( 11 ) 


.908 


Learning Score ( 702 ) 


-.014 (.18) 


-.47(11) 


.650 


Model for (pi) 








Age (yio) 


.025 (.08) 


.18 (13) 


.861 


Model for (P 2 ) 








Gender ( 720 ) 


-.020 (.13) 


-.479 (13) 


.649 


Model for (ps) 








White/non-white ( 730 ) 


-.047 (.12) 


-.93 (13) 


.369 



Random Effects (Var. Components) 


Variance 


df 


Chi-square 


Var. befween courses (Xoo) 


.013 


5 


4.112 (p>. 500) 


Var. in age (xn) 


.001 


7 


4.082 (p > .500) 


Var. in gender (X22) 


.038 


7 


5.760 (p > .500) 


Var. in whife/non-white (X33) 


.040 


7 


5.435 (p > .500) 


Var. within courses (a^) 


3.779 
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